Lesson 1

Introduction to learning

Three Criterion for applying learning

Does a pattern exist
Is it mathematical
Do we have data on this

Learning Problem

Learning problem involves a setup of

Input x
Output y
Target Function f : X -> Y

x is a vector of d dimensions The goal of learning is to approximate the target function. We produce hypothesis function g : X -> Y

A simple hypothesis function is linear hypothesis which approximates learning as hyperplane in d dimensions We assume a 'W' vector

$$h(x) = \sum_{i=1}^{i=d} w_i x_i$$

g(x) = +1 for h(x) > threshold g(x) = -1 for h(x) < threshold

We can simplify this by assuming an element $$x_0 = 1$$, $$w_0 = \text{threshold}$$ $$h(x) = \text{sign} (\sum_{i=0}^{i=d} w_i x_i)$$

Perceptron Learning Algorithm.

The Perceptron learning algorithm is used to decipher the correct values of w

For iterations t = 0,1,2,... Pick a pair (x, y) which is misclassified and $$w(t) = w(t-1) + y_i x_i $$

We will try to classify two digits in mnist dataset.



In [1]:

    
import numpy as np
import matplotlib.pylab as plt

%matplotlib inline



In [2]:

    
from sklearn.datasets import fetch_mldata
mnist = fetch_mldata('MNIST original', data_home='data')



In [3]:

    
X = np.concatenate((mnist.data[0:50], mnist.data[9000:9050]))
Y = np.concatenate((mnist.target[0:50], mnist.target[9000:9050]))
Y[Y != 0] = -1
Y[Y == 0] = 1



In [4]:

    
w = np.random.rand(785)
w_orig = w



In [5]:

    
def h(x):
    v = np.dot(w, np.concatenate(([1], x)))
    if v > 0:
        return 1
    else:
        return -1



In [6]:

    
for _ in range(0, 10000):
    for i in range(0, 100):
        clas = h(X[i])
        if clas != Y[i]:
            w = w + Y[i] * np.concatenate(([1], X[i]))



In [7]:

    
def display(x, label):
    pixels = x.reshape((28, 28))
    plt.title('{label}'.format(label=label))
    plt.imshow(pixels, cmap='gray')
    plt.show()



In [8]:

    
display(X[99], "predicted: %d" % (h(X[99])))



In [9]:

    
display(X[99], "predicted: %d" % (h(X[50])))



In [10]:

    
# Test Accuracy on 12000 inputs
g = []
for i in range(0,12000):
    g.append(h(mnist.data[i]))
G = np.array(g)
accuracy = sum(mnist.target[0:12000] == G)
'Accuracy %d percent' % (accuracy)









    Out[10]:





'Accuracy 94 percent'



In [11]:

    
display(mnist.data[1000], "predicted: %d" % (h(mnist.data[1000])))



In [12]:

    
display(w[1:], "Weight Map")



In [13]:

    
import os
os.system('git add . &&')









    Out[13]:





512